CRITERIA FOR HANKEL OPERATORS TO BE SIGN-DEFINITE 
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Abstract. We show that total multiplicities of negative and positive spectra of a 
self-adjoint Hankel operator H with kernel h(t) and of an operator of multiplication 
by some real function s(x) coincide. In particular, ±H > if and only if ±s(x) > 0. 
The kernel h(t) and its "sign-function" s(x) are related by an explicit formula. 
An expression of h(t) in terms of s(x) leads to an exponential representation of 
h{t). Our approach directly applies to various classes of Hankel operators. In 
particular, for Hankel operators of finite rank, we find an explicit formula for the 
total multiplicity of their negative and positive spectra. 



I. Introduction 
1.1. Hankel operators can be denned as integral operators 



poo 

(Hf)(t)= / h(t + s)f(s)ds (1.1) 
Jo 



in the space L (K+) with kernels h that depend on the sum of variables only. Of 
course H is symmetric if h(t) = h{t). There are very few cases where Hankel 
operators can be explicitly diagonalized. We mention classical results by F. Mehler 
[8], T. Carleman j3], W. Magnus [7] and M. Rosenblum [12] . They are treated in a 
unified way in [16] where some new examples are also considered. 

Our goal here is to find necessary and sufficient conditions for the positivitjfl of 
Hankel operators. This question seems to be of particular importance because of its 
intimate relation to a representation of the function h(t) as the Laplace transform 
of a (positive) measure. Such representations are continuous analogues of moment 
problems. According to the Hamburger theorem (see, e.g., the book [I]) the posi- 
tivity of a discrete Hankel operator is equivalent to the existence of a solution of the 
corresponding moment problem. 
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1 We always use the term "positive" instead of a more precise but lengthy term "non-negative." 
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1.2. Our condition of the positivitjH of Hankel operators is quite explicit. Let B, 

/oo 
b(Z-v)g(v)dv, (1-2) 
-oo 

be the operator in the space L 2 (M) of the convolution with the function 



Of course &(— £) = 6(£) if /i(i) = so that the operator B is symmetric. Our 
main result is that the Hankel operator H > if and only if B > 0. We call &(£) 
the b- function of the Hankel operator (or of the kernel and we use the term 

the sign-function or s- function for its Fourier transform s(x). So our result, roughly 
speaking, means that a Hankel operator H is positive if and only if its sign-function 
s(x) is positive. Note that in specific examples we consider, functions s(x) may be of 
a quite different nature. For instance, s(x) may be a polynomial or, on the contrary, 
it may be a distribution; for example, it may be a combination of delta functions 
and their derivatives. 

Our proofs rely on the identity 

(Hf,f) = (Bg,g) (1.4) 

where 

(K0 = r(l/2 + ifl/(fl=:(H/)(fl, (1.5) 

/(£) is the Mellin transform of f(t) and T(-) is the gamma function. Actually, the 
identity ( 11. 4p or, equivalently, 

H=~*B~ (1.6) 

allows us to find the numbers of positive and negative eigenvalues of a Hankel op- 
erator H. For a self-adjoint operator A, we denote by N + (A) (by N~(A)) the total 
mutiplicity of its strictly positive (negative) spectrum. Then the identity (II. 6p shows 
that 

N±(H) = N±(B). (1.7) 
This result can be compared with Sylvester's inertia theorem which states the 
same for Hermitian matrices H and B related by equation ( II. 6p provided the matrix 
S is invertible. In contrast to the linear algebra, in our case the operators H and B 
are of a completely different nature and B (but not H) admits an explicit spectral 
analysis. 

Of course the results above can be reformulated in terms of the sign-function 
s(x) = a/27t($*6)(o:) where $ is the Fourier transform. Let us introduce the operator 
S = §*B§ of multiplication by s(x). Then (II .6p means that H = where 
E = and dHZJ) means that N±(H) = N±(S). 



2 We usually discuss conditions for H > 0, but of course replacing H by —H we obtain conditions 
for H < 0. 
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1.3. The precise meaning of formula (II. 3p requires some discussion. Observe that 
the denominator in (jl.3p coincides with the numerator for the special case h(t) = e~*. 
It equals T(l — i£) and hence exponentially tends to zero as |£| — > oo. Therefore 

is a "nice" function of £ only under very restrictive assumptions on the kernel 
h(t). Thus to cover natural examples, we have, on the contrary, to extend a class 
of kernels and to work with distributions h(t). The choice of appropriate spaces 
of distributions is also very important. In order to be able to divide in (I1.3P by 
an exponentially decaying function, we assume that the numerator belongs to the 
class of distributions C^°(M)'. It means that the Fourier transform of the function 
9(x) = e x h(e x ) should belong to C£°(WL)'. The corresponding class of functions h(t) 
will be denoted Z' + . Under the assumption h G Z' + we have b G C^°(IR) / . The 
Schwartz space «S(R)' is too restrictive for our purposes which is seen already on the 
example of finite rank Hankel operators. 

The condition $6* G Cg°(R)' for the validity of the identity (II. 4p is very general. It 
is satisfied for all bounded, but also for a wide class of unbounded, Hankel operators 
H. More than that, it is not even required that H be defined by formula (II. ip on 
some dense set. Therefore we work with quadratic forms (Hf, /) which is more 
convenient and yields more general results. In this context it is natural to consider 
distributions h(t) which makes the theory self-consistent. 

If however h G L 1 1 oc (M + ), then 9 G S(R)' and hence $6 G C£°(R)' if 

POO 

/ \h(t)\(l + \lnt\y K dt < oo (1.8) 
Jo 

for some k. Condition (II .8p is quite general; it also does not require that the 
corresponding Hankel operator be bounded. For example, it admits kernels 

h{t) = P{hit)r l (1.9) 

where P(x) is an arbitrary polynomial. Note that Hankel operators with such kernels 
are bounded for P(x) = const only. 

As far as test functions f(t) are concerned, we require that their Mellin transforms 
/ G Cq°(WL). Then both sides of fll.4jl are well defined and the identity holds. We 
note that the distribution &(£) is "worse" than the kernel h(t). On the contrary, due 
to the factor T(l/2 + z£) in (II. 5p . the test function g(£) is "better" than f(t). In 
the case of bounded operators H, this permits us to extend the main identity (11.41) 
to all elements / G L 2 (1R + ). 

1.4. It turns out that the knowledge of the sign-function s(x) allows one to recover 
the kernel h(t) by the formula 

POO 

h{t) = / e- tx h\\)\d\ (1.10) 
Jo 

where 

h\\) = A" 1 s(-lnA). (1.11) 
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Thus h(t) is the Laplace transform of the distribution Xh\X). It is noteworthy that 
tfi G Z' + and the correspondence h i— > is a continuous one-to-one mapping of Z' + 
onto itself. 

Representation (11.101) does not require the positivity of H. If however H > 0, 
then combining our results with the Bochner-Schwartz theorem, we obtain that 
\hr(\)d\ = dm(X) where dm (A) is a positive measure. In this case 

POO 

h(t) = / e- tx dm(\). (1.12) 
Jo 

This representation implies that h(t) is necessarily a completely monotonic function 
and, in particular, h G C°°(R+). Note that the converse statement is also true: a 
completely monotonic function admits representation (11.121) . This is one of famous 
Bernstein's theorems (see his paper [2] , or the book by N. Akhiezer [TJ or the book by 
D. V. Widder [T3] ). In contrast to the Bernstein theorem, we deduce representation 
(I1.12p from the positivity of the Hankel operator with kernel h(t) and show that the 
measure dm(X) satisfies for some x the condition 

POO 

/ (1 + |lnA|)-"A- 1 dm(A) < oo. (1.13) 
Jo 

Hankel operators can also be realized as operators in the space of sequences l+. 
The relation between this discrete representation and the continuous representation 
we consider is given by the unitary transformation of l\ onto L 2 (R + ) constructed in 
terms of the Laguerre polynomials. Thus all our results can in principle be translated 
into the discrete representation. This hopefully will be discussed in another article 
on this subject. 

1.5. A large part of the paper is devoted to applying the general theory to various 
classes of Hankel operators although we do not try to cover all possible cases. In 
some examples the sign definiteness of H can also be verified or refuted with the 
help of Bernstein's theorems. Note however that our approach yields additionally 
an explicit formula for the total numbers of negative and positive eigenvalues of H . 

In Section 5, we present such a formula for Hankel operators of finite rank. Then 
we consider two specific examples. The first one is given by the formula 

h(t) = t k e~ a \ a > 0, k > -1. (1.14) 

Note that the Hankel operator H with such kernel has finite rank for k G Z + only. 
We show that H is positive if and only if k < 0. The second class of kernels is 
defined by the formula 

h(t)=e~ tr , r>0. (1.15) 

It turns out that the corresponding Hankel operator is positive if and only if r < 1. 

Section 6 is devoted to a study of Hankel operators H with non-smooth ker- 
nels. In this case both numbers N±(H) are infinite, and we find the asymptotics of 
eigenvalues of H . 
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Finally, in Section 7 we consider perturbations of the Carleman operator, that 
is, of the Hankel operator with kernel ho(t) = t^ 1 by various classes of compact 
Hankel operators. The Carleman operator can be explicitly diagonalized by the 
Mellin transform. We recall that it has the absolutely continuous spectrum [0, it] of 
multiplicity 2. The Carleman operator plays the distinguished role in the theory of 
Hankel operators. In particular, it is important for us that its sign- function s(x) = 1. 
As was pointed out by J. S. Howland in [6], Hankel operators are to a certain extent 
similar to differential operators. In terms of this analogy, the Carleman operator 
H plays the role of the "free" Schrodinger operator D 2 , D = —id/dx, in the space 
L 2 (IR). Furthermore, Hankel operators H with "perturbed" kernels h(t) = t~ 1 +v(t) 
can be compared to Schrodinger operators D 2 + V(x). The assumption that v(t) 
decays sufficiently rapidly as t — > oo and is not too singular as t — > corresponds 
to a sufficiently rapid decay of the potential V(x) as \x\ — > oo. 

As shown in |T7] , the results on the discrete spectrum of the operator H lying above 
its essential spectrum [0, it] are close in spirit to the results on the discrete spectrum 
of the Schrodinger operator D 2 + V(x). On the contrary, the results on the negative 
spectrum of the Hankel operator H are drastically different. In particular, contrary 
to the case of differential operators with decaying coefficients, the finiteness of the 
negative spectrum of the Hankel operator H is not determined by the behaviour 
of v(t) at singular points t — and t = oo. As an example, consider the Hankel 
operator with kernel 

h(t) = r 1 - 7e~ r , re (0,1). 

Now the kernel of the perturbation is the function which decays faster than any 
power of t -1 as t — ?• oo and it has the finite limit as t — > 0. Nevertheless we show 
that the negative spectrum of H is infinite if 7 > 70 (here 70 = 70(f) is an explicit 
constant) while H is positive if 7 < 70. Such a phenomenon has no analogy for 
Schrodinger operators with decaying potentials. However it occurs for three-particle 
Schrodinger operators and is known as the Efimov effect. 

We also study perturbations of the Carleman operator H by Hankel operators 
V of finite rank. Here we obtain a striking result: the total numbers of negative 
eigenvalues of the operators H = H + V and V coincide. 

As examples, we consider only bounded Hankel operators in this paper. However, 
our general results directly apply to a wide class of unbounded operators, such 
as Hankel operators with kernels (11. 9p . Moreover, with slight modifications our 
method works also for kernels (I1.14p where a > and k is an arbitrary negative 
number. In this case condition fjl.Sp is not satisfied. Hankel operators with kernels 
(II. 9p and dl . 14[) generalize the Carleman operator, and we call them quasi- Carleman 
operators. They will be studied elsewhere. 

1.6. Let us briefly describe the structure of the paper. We obtain the main iden- 
tity (II. 4p and the reconstruction formula (ll.lOp in Section 2. Necessary information 
on bounded Hankel operators (including a continuous version of the Nehari theorem) 
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is collected in Section 3. In Sections 2 and 3 we do not assume that the function 
h is real, i.e., the corresponding Hankel operator H is not necessarily symmetric. 
Spectral consequences of the formula (11.41) and, in particular, criteria of the sign- 
definiteness of Hankel operators are formulated in Section 4. In Sections 5, 6 and 7 
we apply the general theory to particular classes of Hankel operators. 
Let us introduce some standard notation. We denote by $, 

POO 

($«)(£) = (2tt)- 1/2 / u(x)e~ ixi d 



Ix, 

the Fourier transform. The space Z = Z(M>) of test functions is defined as the subset 
of the Schwartz space S = iS(lR) which consists of functions ip admitting the analytic 
continuation to entire functions in the complex plane C and satisfying bounds 

\f(z)\ < C„(l + H)- n e r|Imz| , VzeC, 

for some r = r((p) > and all n. We recall that the Fourier transform $ : Cq°(R) — > 
Z and $* : Z — > Cq°(R). The dual classes of distributions (continuous antilinear 
functionals) are denoted S', C£°(M)' and Z', respectively. In general, for a linear 
topological space £, we use the notation £ for its dual space. The Dirac function 
is standardly denoted S(-). 

We use the notation (•, •) and (•, •) for scalar products and duality symbols in 
L 2 (IR + ) and L 2 (R), respectively. They are always linear in the first argument and 
antilinear in the second argument. The letter C (sometimes with indices) denotes 
various positive constants whose precise values are inessential. 

2. The main identity 

2.1. Let us consider a Hankel operator H defined by equality (11.11) in the space 
L 2 (]R + ). Actually, it is more convenient to work with sesquilinear forms (if/1,/2) 
instead of operators. 

Before giving precise definitions, let us explain our construction at a formal level. 
It follows from (II. ip that 

"OO POO 



POO POO 

{Hh, f 2 )= / h(t + s)f 1 (s)W)dtds 
Jo Jo 

poo 

= / h{t)F~{f)dt =: (h,F), (2.1) 
Jo 



where 



F(t) = / fi(s)h(t - s)ds =: (A * f 2 )(t) (2.2) 







is the Laplace convolution of the functions f\ and A- Formula (12. ip allows us to 
consider h as a distribution with the test function F defined by (12.21) . Thus the 
Hankel form will be defined by the relation 

h[h,f2]=(hj 1 *f 2 ). (2.3) 
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Let us introduce the test function 

Q{x) = F{e x ) =: (TZF)(x) (2.4) 

and the distribution 

9(x) = e x h{e x ) (2.5) 
defined for iGl Setting in ( 12. ID t = e x , we see that 



(h } F}= 9(x)tt(x)dx =: (0,0). (2.6) 

We are going to consider the form ( \2.6\\ on pairs F, ft, such that the corresponding 
test function Q defined by (12.41) is an element of the space Z of analytic functions 
and the corresponding distribution 9 defined by (12. 5p is an element of the dual space 
Z' . The set of all such F and h will be denoted Z + and Z' + , respectively, that is, 

F EZ+ tteZ and h e z' + 9 G Z'. (2.7) 

Of course, the topology in Z + is induced by that in Z and Z' + is dual to Z + . Note 
that h G Z' + if h G L[ oc (M + ) and integral (11.81) is convergent for some k. In this case 
the corresponding function (12. 5p satisfies the condition 



/oo 
\9(x)\(l + \x\)~ K dx < oo, 
-oo 



and hence 9 G S' C Z' . 

Define the unitary operator U : L 2 (IR + ) — > L 2 (W) by the equality 

{Uf){x) = e x ' 2 f{e x ). (2.8) 

Let the set T> consist of functions f(t) such that Uf G Z. Since 

f(t) = t-W(Uf)Qnt) 

and Z C S, we see that functions / G X> and their derivatives satisfy the estimates 

|/( m )(t)| = c n , m r 1 / 2 - m (i + |int|)-" 

for all n and m. Obviously, / G P if and only if (/?(£) = t 1 ^ 2 f(t) belongs to the class 
Z + . 

Let us show that form (I2.3P is correctly defined on functions fi, f 2 G V. To that 
end, we have to verify that function (12. 2p belongs to the space Z + or, equivalently, 
function (12. 4p belongs to the space Z. This requires some preliminary study which 
will also allow us to derive a convenient representation for form (12. 3p . 

Recall that the Mellin transform M : L 2 (IR + ) — > L 2 (IR) is defined by the formula 

POO 

(M/)(0 = (27T)- 1 / 2 / /(/)/ ; 2 'v//. (2.9) 
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Of course, M = $?7 where $ is the Fourier transform and U is operator ( 12. 81) . Since 
both $ and Z7 are unitary, the operator M is also unitary. The inversion of the 
formula fl2-9jl is given by the relation 

/oo 
m)t' 1/2+ M, f = Mf. (2.10) 
-oo 

Let T(z) be the gamma function. Recall that T(z) is a holomorphic function in 
the right half-plane and T(z) ^ for all z G C. According to the Stirling formula 
the function T(z) tends to zero exponentially as \z\ — > oo parallel with the imaginary 
axis. To be more precise, we have 

r(a + iX) = e" (2a - 1)/4 (27r/e) 1/2 A Q - 1/2 e iA(ln (l + 0(A -1 )) (2.11) 



for a fixed a > and A — > +oo. Since T(a — zA) = T(a + iX), this yields also the 
asymptotics of T(a + iX) as A — > — oo. 

If G £>, j = 1, 2, then £ = Mfj = $Ufj G C£°(M) and hence the functions 

ffj (0 = r(i/2 + i0^(0, i = 1,2, (2.12) 

also belong to the class C£°(R). Let us introduce the convolution of the functions 
9i and 92, 

poo 

(0i*02)(O=/ 9i(C - V)92{r])dr], 



and set 

(Jg)(t) = g(-t). 

We have the following result. 

Lemma 2.1. Suppose that fj G T>, j = 1,2, and define functions gj(£) by equality 
(I2.12p . Let the function Q(x) be defined by formulas (I2.2p and (12.41) . Then 

($n)(0 = M-^rxi + itrWgi) * g 2 )(0- (2.13) 

Proof. Substituting (12.101) into definition (12.21) , we see that 

pi poo poo 

F(t) = (2n)- 1 / ds / fi(r)(t-s)- 1/2 - iT dT f 2 (a)s^ 2+l<7 da. 

Jo J — oo J — oo 

Observe that 

[\t - . T v™r.-Wi. = ^-^(1/2 -^(1/2 + ^) 
7 T(l + t(a - r)) 

Then using definition ( I2.12p we obtain the representation 

/OO /"OO 
/ f ^r(l + i(a - T))- l ^)g 2 (a)drda 
•00 •/ —00 

/oo 
f*r(i + ifl- l ((^i)*(fe)(Ode, 
-00 
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whence 

/oo 
e i ^T(l + i^- 1 ((Jg 1 )*g 2 mdC 
-oo 

This is equivalent to formula f 1 2 . X 3 j) . □ 

Observe that the function T(l + i£) _1 in the right-hand side of (I2.13P tends to 
infinity exponentially as |£| — > oo. Nevertheless $0 e C^°(R) because (Jg~i) * gi G 
C °°(M) for gi,g 2 e Cg°(K). Thus we have 

Corollary 2.2. Le£ fj G P, j = 1,2, and Zei £/ie function Q(x) be defined by 
formulas (12. 2p and (12.41) . TTien Q E Z or, equivalently, F G 2+ . 

Now we are in a position to give the precise definition. 

Definition 2.3. Let h G -2^ and / 3 - £ V, j = 1,2. Then the Hankel sesquilinear 
form is defined by the relation (12.31) . 

We shall see in subs. 2.4 that h G Z' + is determined uniquely by the values 
(h, ft * f 2 ) on f Xl f 2 G V, that is, h = if (/i, £ * / 2 > = for all / x , / 2 G V. 
Of course definition (12. 3p can be rewritten as 

h\f u f 2 ] = (e,n) (2.i4) 

where 

n(x) = (f 1 *f 2 )(e x ) 

and 6* is distribution (12. 5p . 

We sometimes write h[fi, f 2 ] as integral (12.11) keeping in mind that its precise 
meaning is given by Definition 12.31 

2.2. Our next goal is to show that (12.141) is the sesquilinear form of the convolution 
operator B, that is, it equals the right-hand side of (II ,4p . Here the representation 
of Lemma 12.11 for the function 

G(0 = V&(l + iO($fi)(0 (2.15) 

plays the crucial role. 

Since 9 G Z\ its Fourier transform a = §6 is correctly defined as an element of 
Co°(R)'. Formally, 

POO 

o(0 = ($6»)(0 = (27T)- 1 / 2 / fc^df, (2.16) 

./O 

that is, a(£) is the Mellin transform of the function h{t)t 1 / 2 . Let UgZ. Passing to 
the Fourier transforms and using notation (12.151) . we see that 

(0, n) — (a, ®Q) — (b, G) (2.17) 

where G G Cq°(M.) and the distribution b 6 C^°(M)' is given by the relation 

6(0 = (27r)- 1 / 2 a(Or(l-^)" 1 (2-18) 
which is of course the same as (11.31) . Thus we are led to the following 
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Definition 2.4. Let h G Z' + . The distribution b G Cq°(M)' defined by formulas 
( T23j) . Q7TB|) and f[2TTgj) is called the fe-function of the kernel h(t) (or of the Hankel 
operator H). Its Fourier transform s = ^/2n&*b G Z' is called the s-function or the 
sign-function. 

Recall that the distribution h? G Z'^ was defined by relation (11. lip . The following 
assertion is an immediate consequence of formulas (12. 5p . (12 . 1 6[) and (I2.18p . 

Proposition 2.5. The mappings 

yield one-to-one correspondences (bijections) 

z' + ^z' ->■ c °°(M) c?{m) z' ->■ z;. 

^4// o/ them, as well as their inverse mappings, are continuous. 
Putting together equalities (I2.6P and (I2.17p . we see that 

(h,F) = (b,G). (2.19) 
Combining this relation with Lemma [27T1 and Definitions I2.3[ I2T41 and using notation 



(II. 5p . we obtain the main identity (II. 4p . To be more precise, we have the following 
result. 

Theorem 2.6. Suppose that h G Z' + , and let b G C^°(1R) / be the corresponding b- 
function. Let fj G T>, j = 1, 2, and let the functions gj = Efj be defined by formula 
(I2.12p . Then gj G Cq°(IR) and the representation 

(h, h * f 2 ) = (b, (Jgi) * g 2 ) =■■ b[ gi , g 2 ] (2.20) 

holds. 

Passing in the right-hand side of (I2.20p to the Fourier transforms and using that 

^((J9i) * 92) = (27r) 1/2 Wg~ 1 ^g 2 , 

we obtain 

Corollary 2.7. Let s G Z' be the sign-function of h, and let Uj = &*gj = &*^fj G 
Z. Then 

(h,fi*f 2 ) = (s,UxU 2 ) =: s[ui,u 2 ). (2.21) 
Loosely speaking, equalities (I2.20p and (I2.2ip mean that 

« °° (2-22) 



s(x)ui(x)u 2 (x)dx . 
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In the particular case h(t) = we have 8(x) = 1, 

a(0 = (2tt) 1 / 2 5(0, 6(0 = *(£). *(*) = !, (2.23) 
and hence (12.221) yields 

/OO /"OO 
giititoffidt = / |r(i/2 + zOI 2 A(0/ 2 m, 
OO J — OO 

where 

|r(l/2 + iOI 2 = jjh?v (2-24) 

COSh(7T4) 

This leads to the familiar diagonalization of the Hankel operator H with kernel 
hit) = t~ l . This operator is known as the Carleman operator and will be denoted 
by C. 

2.3. According to Proposition 12.51 the distribution w determines uniquely the 
distribution h. Let us now obtain an explicit formula for the mapping i — y h. This 
requires some auxiliary information. 

Let T Q : Co°(M) -)■ Cg°(R), a > 0, be the operator of multiplication by the 
function T(a + Making the change of variables t = e~ x in the definition of the 
gamma function, we see that 

roc /*oo 

r(a + iX) = / e-H a+iX - l dt = / e- e ~ x e- ax e- ixX dx, a > 0, 

JO J -oo 

and hence 

/OO 
e ixX T(a + iX)dX = e~ e ~* " e~ ax . (2.25) 
-OO 

It follows that 

/OO 
e «<v-*) e -#-Sl(y)dy. (2.26) 
-OO 

Let us also introduce the operator L Q : 

POO 

(UF)(A) = X a e-'H^F^dt, A > 0, a > 0. 

Obviously, L Q F G C°°(R + ) for all bounded functions F(t) and, in particular, for 
F G Z + . Note that L Q is the Laplace operator L, 

POO 

(LF)(A) = / e~ tx F(t)dt, (2.27) 
Jo 

sandwiched by the weights X a and t a ~ l . 

Recall that the operator 1Z defined by (12.41) is a one-to-one mapping of Z + onto 

Z. 

We need the following result. 
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Lemma 2.8. For all a > 0, the identity 

|_ Q = n^J&TJ&R. (2.28) 

holds. In particular, L a as well as its inverse are the one-to-one continuous mappings 
of Z + onto itself. 

Proof. Putting Q(y) = (TZF)(y) = F(e y ) in (12.261) and making the change of vari- 
ables t = e y , we find that 

POO 

(®*r a ®1ZF) (x) = e~ ax / e- e ~ x H a - l F(t)dt. 

Jo 

Now making the change of variables A = e~ x , we arrive at the identity (12.281) . 

Consider the right-hand side of (12.281) . All mappings 1Z : Z + — > Z, $ : Z — > 
C °°(M), T a : C °°(M) ->• Cq°(R), : C^°(R) Z, J : Z Z are bijections. All of 
them as well as their inverses are continuous. Therefore the identity (12.281) ensures 
the same result for the operator L a : Z+ — > Z + . □ 

To recover h(t), we proceed from formula f)2.19p . Passing to the Fourier trans- 
forms, we can write it as 

(h,F) = (2nr 1 / 2 (s,^G) 

where G is defined by formulas ([H]), fl2TT5|) . that is, G = (27r) 1 / 2 r 1 $^F. Therefore 
using the identity (12.28P for a = 1, we see that 



(h,F) = (s,JTZl l F}= / 

J — c 



s(x)(L 1 F)(e- x )dx. 



Making the change of variables A = e x in the right-hand side, we obtain the identity 

(h,F) = (h\l 1 F). 

Passing here to adjoint operators and using that F G Z + is arbitrary, we find that 

h = L\h\ (2.29) 

which gives the precise sense to formula (ll.lOp . Of course formula (ll.lOp can also 
be rewritten as 

/oo 
e- te ~ x e- x s(x)dx. 
-oo 

Let us state the result obtained. 

Theorem 2.9. Let h G Z' + , and let s G Z' be the corresponding sign-functions (see 
Definition al^ . Define the distribution by formula (11.111) . Then h\\) belongs to 
the class Z' + and h can be recovered from hfi by formula (I2.29P . 

We emphasize that in the roundabout h i— )■ h-> h the mappings h i— )■ as well 
as its inverse are one-to-one continuous mappings of the set Z' + onto itself. 

Let us also give a direct expression of u(x) = ($*g)(x) in terms of f(t). 
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Lemma 2.10. Suppose that f G T> and put cp(t) = t l l 2 f(t). Let g(£) be defined by 
formula ( II .5|) and u(x) = ($*g)(x). Then 

u(x) = (L 1/2 ip)(e- x ). (2.30) 

Proof. Since (lZ(p)(x) = (Uf)(x), it follows from formula f !2.28|) for a = 1/2 that 

(Jl^ 1 J&*T 1 /2&U f)(\) = (L 1/2¥ >)(A). 

The left-hand side here equals (TZ~ 1 J'u)(X) which after the change of variables A = 
e~ x yields f[2T30l . □ 

Now we can rewrite identity (I2.2ip in a slightly different way. 

Corollary 2.11. Let h G Z' + , and let the distribution hr G Z' + be defined by formula 
(11. lip . Then for arbitrary fj G T>, j — 1, 2, and (fj(t) = t 1 ^ 2 fj(t), we have 

(hji*f 2 ) = (h\L^Li /2 <P2)- (2-31) 

Proof. It suffices to make the change of variables x = — In A in the right-hand side 
of (I2.22p and to take equality (I2.30p into account. □ 

We emphasize that according to Lemma I2.8[ \-i/2 ( Pj £ Z + and hence 



l_i/2<^iLi/2y?2 G Z + . Thus the right-hand side of (I2.3ip is correctly defined. 

2.4. Finally, we check that a distribution h G Z', is determined uniquely by the 
values (h, fi * f 2 ) on fx, f 2 G V. First we consider convolution operators. Let us 
introduce the shift in the space L 2 (R): 

{T(T)g)(Z) = g(Z-T), tgR. (2.32) 

Since 

/oo 
(T(r) gi )(092(r)dT, \/ 9l ,g 2 G Cg°(R), 
-oo 

we have the formula 

/oo 
(b,T(r)J gi )g^)dr (2.33) 
-oo 

where for b G (M)' the function (b, T(r)j7"^i) is infinitely differentiable in r 6 1. 
The following assertion is quite standard. 

Lemma 2.12. Let b G C£°(R)'. Suppose that (b, {Jgi) * g 2 ) = for all g x ,g 2 G 
C^(R). Thenb = 0. 

Proof. If (b, (Jgx) * g 2 ) = for all g 2 G C °°(M), then (b,T(r)Jgi) = for all r G K 
according to formula (12.331) . In particular, for r = we have J7gi) = whence 
6 = because g± G C^°(M) is arbitrary. □ 

Next we pass to Hankel operators. 
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Proposition 2.13. Let h G Z' + . Suppose that (h, fx * f 2 ) =0 for all fi,f 2 € V. 
Then h = 0. 

Proof. Let b G C£°(M)' be the 6-function of h (see Definition 12. 4p . For arbitrary 
9i, 92 ^ Co°(^); we can construct fi, f 2 G T> by formula (12.121) . Since (/t, fi*f 2 ) = 0, 
it follows from the identity (I2.20p that (b, {Jgi) *gz) = 0. Therefore 6 = according 
to Lemma [2.121 Now Proposition 12.51 implies that h = 0. □ 

3. Bounded Hankel operators 

Our main goal here is to show that the condition h G Z' + is satisfied for all 
bounded Hankel operators H. 

3.1. In this section we a priori only assume that h G C^°(lR + ) / and consider the 
Hankel form (T23D on functions fx,f 2 G C °°(M + ). Let T + (r) where r > be the 
restriction of the shift (I2.32p on its invariant subspace L 2 (IR + ). Since 

/"OO 

(/i*/ a )(f)=/ {T + {r)h){t)f 2 {T)dr, V/ 1 ,/ 2 GC CO (R + ), 
Jo 

for all h G C^°(lR + ) / we have the formula 

POO 

to/i*/2>=/ (KT + {r)h)Mr)dT. (3.1) 

Here the function (/t, T + (r)/i) is infinitely differentiable in r G R+, and this function, 
as well as all its derivatives, have finite limits as r — > 0. In the theory of Hankel 
operators, formula (13. ip plays the role of formula f !2.33|) for convolution operators. 
The proof of the following assertion is almost the same as that of Lemma 12.121 

Proposition 3.1. Let h G C^°(lR + ) / . Suppose that (h, fi * f 2 ) — for all f 1: f 2 G 
C °°(M + ). Then h = 0. 

Proof. If {h, ft*f 2 ) = for all f 2 G Cq 00 ^), then (/i,T+(r)/ 1 ) = for all r G [0, oo) 
according to formula (13. ip . In particular, for r = we have {h, fi) = which implies 
that h = because f\ G C^°(1R + ) is arbitrary. □ 

Of course Propositions 12.131 and 13.11 differ only by the set of functions on which 
the Hankel form is considered. 
Assume now that 

\(h,f*f)\ <qi/|| 2 , V/GC °°(M + ). (3.2) 
Then there exists a bounded operator H such that 

(Hf 1 J 2 ) = (hJ 1 *f 2 ), V/ 1 ,/ 2 GC CO (R + ). (3.3) 
We call H the Hankel operator associated to the Hankel form (h, fi * f 2 ). 
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3.2. It is possible to characterize Hankel operators by some commutation rela- 
tions. Let us define a bounded operator £ in the space L 2 (M + ) by the equality 

(S/)(t) = -2e- t f e s f(s)ds. 
Jo 

Note that 

POO 

£ = -2 / T + (r)e- T dT. (3.4) 



Lemma 3.2. Zet assumption (13. 2 p /io/d T/ien t/te corresponding Hankel operator 
H satisfies the commutation relations: 



and 



Proof. Since 



HT + (r)=T + (r)*H, Vr > 0, (3.5) 
= £*#. (3.6) 



(T+(r) A) * / 2 = A * (T + (r)/ 2 ), Vr > 0, 

relation (13. 5 p directly follows from definition (13. 3p . By virtue of formula (13 ,4p . 
relation (13. 6 p is a consequence of (13. 5p . □ 



Below we need the Nehari theorem; see the original paper [9], or the books [TO] . 
Chapter 1, §1 or [UJ, Chapter 1, §2. We formulate it in the Hardy space H 2 (R) 

of functions analytic in the upper half-plane. We denote by £ the operator of 
multiplication by the function [jj, — + i) in this space. 

Theorem 3.3 (Nehari). Let u G L°°(IR) 7 and let a bounded operator H in the space 
H^_(]R) be defined by the relation 

/oo 
w(//)/i(-//)/ 2 Md/x, VA, A e H^(R). (3.7) 
-oo 

Then HTj = Y>*H . Conversely, if H is a bounded operator in H^(R) and HT, = T,*H , 
then there exists a function uj G L°°(R) such that representation (13. 7p holds. 

The following assertion can be regarded as a translation of this theorem into the 
space L 2 (M + ). Recall that the Fourier transform $ : H^_(R) — > L 2 (R + ) is the unitary 
operator. Since 

r ^i e -^ = _ 47re -* 

for t > and this integral is zero for t < 0, we have the relation 

S = (3.8) 
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Theorem 3.4. If representation ( 13. 3j) holds with h = $w where u G L°°(R) (in this 
case h G S' C C^°(R + )'), t/ien estimate (13.21) zs true and t/ie corresponding Hankel 
operator satisfies commutation relation (13. 5p . Conversely, if a bounded operator H 
satisfies (13. 5p . £/ien representation (13.31) /io/cfe u;ii/i /i = $w /or some a; G L°°(R). 

Proof. Since 

(**(/i*/ 2 ))(aO = (JhmTM, VA./a G CS°(K+), 

where A = A = $*/2 and (Jfc)(n) = we have 

(/i,/i*/2>=(^/i,(J70/2>, V/i6 5'. (3.9) 

Therefore estimate (13. 2p is satisfied if G L°°(R). Relation ( 13. 5 p for the corre- 
sponding Hankel operator if follows from Lemma 13.21 

Conversely, if a bounded operator H satisfies relation (I3.5p . then by virtue of (13.41) 
it also satisfies relation (13. 6p . Hence it follows from ( 13. 8 j) that i/£ = £*i/ where 
if = $*i/$ is a bounded operator in the space H^(R). Thus, by Theorem I3.3[ there 
exists a function uj G L°°(R) such that representation (13. 7p holds. It means that 

/oo 
u;(/i)A(-//)/ 2 (/i)rf/i, V/ 1; / 2 GL 2 (R + ). (3.10) 
-oo 

If h = §cu, then the right-hand sides in (13.91) and (I3.10p coincide. This yields 
representation (13.31) . □ 

Corollary 3.5. For a bounded operator H , commutation relations (13.51) and (13.61) 

are equivalent. 

Proof. As was already noted, (13. 6p follows from (13.51) according to formula (13.41) . 
Conversely, if H satisfies (13.51) . then representation (13.31) holds according to Theo- 
rem [3]4l Thus it remains to use Lemma [3.21 □ 

A function u G L°°(R) such that = h is called the symbol of a bounded Hankel 
operator H with kernel h(t). Of course if to G Hf°(R), that is, u admits an analytic 
continuation to a bounded function in the lower half-plane, then the corresponding 
Hankel operator is zero. Therefore the symbol is defined up to a function in the 
class H~(R). 

3.3. Now we are in a position to check that the condition h G Z' + is satisfied for 
all bounded Hankel operators. By definition ( 12. 7p it means that distribution (12. 5p 
belongs to the class Z' . We shall verify the stronger inclusion 9 G S' . 

To that end, it suffices to check that, for some N G Z + and some k G R, 

jv 

< C^max((l + \x\) K \Q {n) {x)\), Vfi G C °°(R). (3.11) 

n=0 X£ 
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Putting F(t) = Q(lnt), we see that (13. lip is equivalent to the estimate 

JV 

\(h,F)\ < C^max ((1 + \lnt\rt n \F^(t)\), F G C™(R + ). (3.12) 

n=0 

If h G Lj, c (M), then estimate (13. 12[) for A^ = is equivalent to the convergence of 
integral fll.8p . If H is Hilbert-Schmidt, that is 

POO 

/ \h(t)\Hdt < oo, 
Jo 

then integral (11.8p converges for any k > 1/2. Similarly, if \h(t)\ < Ct^ 1 , then 
integral ( 11. 8ft converges for any /c > 1. 

For the proof of (13 . 12[) in the general case, we use the following elementary result. 
Its proof is given in Appendix A. 

Lemma 3.6. // F G Cq°(R + ) ; then for an arbitrary k > 2 the estimate 

2 

\\®*F\\ LHm < C^max ((1 + | lnf|)«t»|FW(f)|) 

n=0 + 

Corollary 3.7. If h = $w w/iere w G L°°(R), then estimate (KWf holds for N = 2 
and an arbitrary k > 2. 

Since, by Theorem 13.41 for a bounded Hankel operator H, its kernel h = $Co> for 
some oo G L°°(R), we arrive at the following result. 

Theorem 3.8. Suppose that h G C^l^)' and that condition ( 13. 2 p is satisfied. 
Then estimate ( I3.12p holds for N = 2 and an arbitrary k > 2; in particular, h G 

The following simple example shows that for N = estimate ( I3.12p is not in 
general true (for all k). 

Example 3.9. Let h(t) = e~ lf2 . Then the corresponding Hankel operator H is 
bounded because according to the formula e~ l ^ t+s ^ = e~ lt e - l2ts e ~ ls ; it is a product 
of three bounded operators. However integral ( 11. 8ft diverges at infinity for all k. In 
this example condition (I3.12p is satisfied for A^ = 1 and n = 0. Indeed, integrating 
by parts, we see that 

/ h(t)Fjt)dt = - hi{t)F'{t)dt (3.13) 
Jo Jo 

where the function h\{t) = J *e _is2 ds is bounded and h\{t) = 0(t) as t — > 0. 
Therefore integral (I3.13P is bounded by max (e t + ((1 + | \nt\) K t\F'(t)\) for k > 1. 
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Note that the symbol of H equals u(fi) = (27r)~ 1 e~ 7ri / 4 e lAl2 / 4 . More generally, one 
can consider the class of symbols co(fi) such that uj G C°°(R), u;(/x) = e luJotJ,a , u > 0, 
for large positive /i and = for large negative \i. Of course Hankel operators 
with such symbols are bounded. Using the stationary phase method, we find that 
for a > 1 the corresponding kernel h(t) has the asymptotics 

h(t) ~ h Q t p e lat \ t->oo, (3.14) 

where /3 = (1 — a/2) (a — 7 = a(a — and h , a = a are some numbers. 

Moreover, is a bounded function on all finite intervals. Similarly to Example 13.91 
it can be checked that for such kernels condition (13.121) is satisfied for iV = 1 but 
not for N = 0. If a G (0, 1), then h(t) has asymptotics (13.141) for t -» 0. 

3.4. Here we shall show that, for bounded Hankel operators H, the represen- 
tations m and f l2~2Tj) extend to all / 1( / 2 G L 2 (M+). By Theorem ESI we have 
h G 2+. Let b and s be the corresponding b- and s-functions (see Definition 12. 4p . 
Recall that the operator S : L 2 (IR + ) — >■ L 2 (R) was defined by formula (II. 5p . We de- 
note by K the operator of multiplication by the function A/cosh(7r^)/7r in the space 
L 2 (IR). It follows from identity (I2.24p and the unitarity of the Mellin transform (12.91) 
that 

= ||/||, 

and hence the operator KS : L 2 (W + ) — > L 2 (IR) is unitary. Therefore in view of the 
identities (12.201) and (12.211) we have the following result. 

Lemma 3.10. The inequalities (13 ,2ft . 

|(M^*S>|<C1|i^|| 2 , V^GC °°(M), (3.15) 

and 

\(s, \u\ 2 )\ < C\\K<S>u\\ 2 , VnGZ, (3.16) 

are equivalent. The Hankel operator corresponding to form (12.31) is bounded if and 
only if one of equivalent estimates (13.21) . (13.151) or (I3.16P is satisfied. 

These estimates can be formulated in a slightly different way. Let us introduce 
the space S C L 2 (M) of exponentially decaying functions with the norm \\g\\e = 
\\Kg\\. Then the space W = &*£ consists of functions u(x) admitting the analytic 
continuation u(z) in the strip Imz G (— 7r/2, 7r/2); moreover, functions u(x + iy) 
have limits in L 2 (R) as y -»■ ±vr/2. The identity 

/oo 
(Kz + ztt/2)| 2 + |m(x - m/2)| 2 )rfx =: H^H^y 
-oo 

defines the Hilbert norm on W. We call W the exponential Sobolev space because it 
is contained in standard Sobolev spaces H'(IR) for all I. The operators H : L 2 (M + ) — > 
S and E := $*S : L 2 (IR + ) — > W are of course unitary. Obviously, \\Kg\\ and 
in the right-hand sides of (13.15P and (13.161) can be replaced by \\g\\s and ||w||w> 
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respectively. Note that the inclusions / G L 2 (R + ), g = H/ G S and u = H/ G W 
are equivalent. 

If one of the equivalent estimates (13. 2p . (13 . 1 5[) or (13. 16[) is satisfied, then all oper- 
ators H : L 2 (R) — >■ L 2 (R), B : S ^ S' and S 1 : W — > W are bounded. Using that 
relations /„ -»> / in L 2 (R+), = 5/ n -»■ g = 5/ in £ and w n = -+ u = $*g 
in W are equivalent, we extend f[2T20]) and flZgT]) to all / G L 2 (M + ). Thus we have 
obtained the following result. 

Proposition 3.11. If one of equivalent estimates (13.21) . (I3.15P or (I3.16P is satisfied, 
then the identities 

(Hfu f 2 ) = (Bg u g 2 ) = (Su u u 2 ), gj = Ef j} Uj = 

are true for all f\, f 2 G L 2 (R + ). 

Let Ki be the operator of multiplication by the function (l+£ 2 ) 1 / 2 . Then estimates 
(I3.15P or (I3.16P are satisfied provided 

\(b,{Jg)*g)\<C l \\K l g\\ 2 or \(s,\u\ 2 )\ < QWuf^ (3.17) 

for some I; in this case 

C = Qvrmax ((1 + ^ 2 ) i (cosh(7rO) _1 ) . 

We note the following assertion. 

Proposition 3.12. A Hankel operator H is bounded if its sign-function satisfies the 
condition 

s G L 1 (R) + L°°(R). (3.18) 
If s G L°°(R) and s(x) — > as \x\ — > oo, then H is compact. 

Proof. The first statement is obvious because under assumption (13.181) the second 
estimate (I3.17P is satisfied for / > 1/2. To prove the second statement, we observe 
that the operator S^K^ 1 is compact because both S and K~ l are operators of 
multiplication by bounded functions which tends to zero at infinity. It follows that 
the operator 

is also compact. □ 

We emphasize, however, that as show already examples of Hankel operators H of 
finite rank (see formula (15.91) for h(t) = e~ at ), condition (13.181) is not necessary for 
the boundedness of H. 
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4. Criteria of the sign-definiteness 

In this section we suppose that h{t) = h(t) so that the operator H is symmetric. 
The results of Section 2 allow us to give simple necessary and sufficient conditions 
for a Hankel operator H to be positive or negative. Moreover, they provide also 
convenient tools for a evaluation of the total multiplicity of the negative and positive 
spectra of H. We often formulate results only for the negative spectrum. The 
corresponding results for the positive spectrum are obtained if H is replaced by 
—H. 

4.1. Actually, we consider the problem in terms of Hankel quadratic forms rather 
than Hankel operators. This is both more general and more convenient. As usual, 
we suppose that a distribution h G Z' + and introduce the b- function b G C^°(R) / 
and the s-function s G Z' by Definition 12.41 

Below we use the following natural notation. Let h[tp, ip] be a real quadratic form 
defined on a linear set D. We denote by iV±(h) the maximal dimension of linear sets 
M.± C D such that ±h[<p, ip] > for all ip G M.±, <p ^ 0. We apply this definition to 
the forms h[f, /] = {h, /*/) defined on V, to b[g, g] = (b, Jg * g) defined on C£°(R) 
and to s[u, u) = (s, \u\ 2 ) defined on Z. Of course, if D is dense in a Hilbert space % 
and h[(p, (p] is closed on D, then for the self-adjoint operator H corresponding to h, 
we have N±(H) = N±(h) . 

Observe that formula (11.51) establishes one-to-one correspondence between the sets 
D and C^°(1R). Moreover, the Fourier transform establishes one-to-one correspon- 
dence between the sets C^°(IR) and Z. Therefore the following assertion is a direct 
consequence of Theorem 12.61 

Theorem 4.1. Let h e Z' + . Then 

N±(h) = N±(b) = N±{s). 

In particular, we have 

Theorem 4.2. Let h G Z' + . Then the form ±{h, f * f) > for all f G V if and 

only if the form ±{b, Jg * g) > for all g G C^°(1R) ; or the form ±(s, \u\ 2 ) > for 
all u G Z. 

4.2. In many cases the following consequence of Theorem 14.11 is convenient. 
According to Proposition 13.121 under the assumptions of Theorem I4.3[ H is defined 
as the bounded self-adjoint operator corresponding to the form (h, f * f). Therefore 
N±(h) = N±(H) is the total multiplicity of the (strictly) positive spectrum for the 
sign "+" and of the (strictly) negative spectrum for the sign "— " of the operator 
H. For definiteness, we consider the negative spectrum. 

Theorem 4.3. Let h G Z' + , and let the corresponding sign-function satisfy condition 
(I3.18p . If s(x) > 0, then the operator H is positive. If s(x) < —Sq < for almost 
all x in some interval Aci, then the operator H has infinite negative spectrum. 
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Proof. If s(x) > 0, then H > according to the second relation ( 12.22ft . 

Let s(x) < —Sq < for x G A. For an arbitrary N, we shall construct a linear set 
£ C Z of dimension iV such that s[u, u] < for all u G £, u ^ 0. Then the second 
statement will follow from Theorem 14.11 

Choose a function ip G Co°(R) such that ip(x) = 1 for x G [—5,(5] and </?(x) = 
for x ^ [—25, 25] where <5 = (5 at is a sufficiently small number. Let points ctj G A, 
j = 1, . . . , N, be such that ay+j — ay = ay — ay-i for j = 2, . . . , iV — 1. Set 
Aj = (oij — 5,(Xj + (5), Aj = [otj — 25, a j + 25). For a sufficiently small 5, we 
may suppose that Aj C A for all j = 1,...,N and that Aj+i C\ Aj = for 
j = 1, . . . , N — 1. We set <Pj(x) = ip(x — aj). Since s(x) < — so < for x G A, we 
have 



oo 

2 



( S} \ifj\ 2 } = / s(x)|^(x)| 2 c/x < -25s < 0. (4.1) 

J — oo 

The functions ipi, . . . ,<px have disjoint supports and hence (s, |w| 2 ) < for an arbi- 
trary non-trivial linear combination u of the functions tpj. 

The problem is that (fj G C^°(R) but (pj Z (and even <pj (jL W). Thus we have 
to approximate these functions in the topology of S by functions ip^ G Z. For a 
given ft > 1/2 and an arbitrary e > 0, we can find a function cp( e > G such that 

max ((1 + \x\) K \ip(x) - tp {e) (x)\) < e. (4.2) 

Now we set <p^\x) = (p^ £ \x — aj). 

Let us check that for e small enough, the functions ip^\ . . . , ip^ are linearly 
independent. Assume that 

N 

$>^f(*)=0. (4.3) 

If a; G Aj(., then \<p£ \x) — 1\ < e and |<^- (x)| < e for j 7^ fc. Therefore ( 14. 3 p yields 
the estimate 

(l-e)|A fc | <£^l A il" 
Summing these estimates over k = 1, . . . , N, we see that 

N N 

(1 -e)£|Ail £|Ail- 
3=1 3=1 

Hence Xj = for all j = 1, . . . , N if e(N - 1) < 1 - e. 
It follows from gZQ, QOJ that, for all j = 1, ... , N, 

00 

s(x)|^ £) (x)| 2 rfx < -2c5s + CV 
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and that 



XjXk / s(x)tpf\x)<p®{x)dx\ < Ce(|A/ + \X k \ 2 ), j ^ k. 



j 

Thus we have the estimate 

noo N N 

/ s(x)\Y,^ { ;\x)\ 2 dx<-J2\^\ 2 ^s -Ce). (4.4) 

J -°° 3=1 3=1 

The right-hand side here is negative if e is small enough and Ylf=i \ ^j\ 2 0- 

Let £ be spanned by the functions (p\ , . . . ,(p^ for sufficiently small e. Then 
dim£ = N and s[u, u] < for all u G C, u ^ according to (14.41) . It follows from 
Theorem O that nJ{H) > N, and hence N_(H) = oo. □ 

Theorem 14.31 can be reformulated, although in a weaker form, in terms of the 
functions &(£) and even h(t). Suppose, for example, that 

beL\m). (4.5) 

Then its Fourier transform s{x) is a continuous function which tends to as |x| — > oo. 
The operator B defined by formula (II. 2p is bounded, self-adjoint and 

spec(-B) = [min six), maxs(x)l. 
xes. xm 

The result below follows directly from Theorem 14.31 Note that by Proposition 13.121 
under assumption ( 14. 5 p the operator H is compact. 

Proposition 4.4. Under assumption (14.51) the Hankel operator H is positive if and 
only if s(x) > 0. If min xe ]R s(x) < 0, then necessarily H has an infinite negative 
spectrum. 

In particular, condition (14. 5 j) is satisfied if 

h(t) = d{l * t] where 6 G Z. 

In this case a = <&9 G C£°(R) and hence b G C^(R). 

4.3. For a proof that a Hankel operator is not sign-definite it is sometimes even 
not necessary to calculate the sign-function s(x) (the Fourier transform of It 
turns out that if &(£) grows as |£| — > oo, then the form b[g, g] = (b, Jg * g) cannot be 
sign-definite. More precisely, we have the following statement about convolutions 
with growing kernels 6(— £) = &(£). 



" oo / 
J \oc\ 



Theorem 4.5. Letb = 6o+^oo where b G C p iW}' for some p G Z + andb^ G LJ; 
Suppose that there exists a sequence of intervals A n = (r n — a n ,r n + o~ n ) where 
r n — > oo (or equivalently r n — > — oo) anc? t/ie sequence a n is bounded such that 

lim cr^ min ReOoo(£) = oo or lim a l n max Re (£) = —00, (4.6) 

n— i>oo §£A n n— >oo £eA n 
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where I = 2 if p = or p = 1 and I = p + 1 if p > 2. JTien /or both signs N±(b) > 1. 

Proof. Since 6 can be replaced by —b, we can assume that, for example, the first 
condition (14. 6[) is satisfied. Pick a real even function (p e Cq°(R) such that <£>(£) > 0, 
p(0 = 1 for |f | < 1/4, = for |f | > 1/2 and set 

9n(0 = <p({£ - r n /2)/a n ) ± + r n /2)/a n ). (4.7) 

An easy calculation shows that 

((^n) * <7n)(0 = 2o-„^(^/o-„) ± <7 n V((£ - ^n)/^n) ± ^((f + r n )/«7 n ) (4.8) 

where ^ = (J(p)*(p G C °°(R). The function ^(f) is also even, > 0, V(0 > 1/8 
for |f | < 1/8 and = for |f | > 1. 

Since \(b ,g)\ < C\\g\\cp, it follows from (14.81) that 

\(b ,(J9n)*9n}\<Ca 1 r r P . (4.9) 
Moreover, again according to (14.81) we have 

/OO /*OG 
b 00 (a n r])ip(r])dr]±2al / Re 6 00 (cr n ?7 + r n )ij)(ri)dri. 
-OO 1/ —00 

(4.10) 

The first term in the right-hand side is O(cr^). For the second one, we use the 
estimate 

/OO 
tte&«„(a n 77 + r n M77)d77> min Re MO- (4.11) 
-oo |S-r n [<o-„ 

Let us first choose the sign " + " in (14.71) . Then using representation (I4.10p and 
putting together estimates (14.91) and (14. lip , we obtain the lower bound 

(&, {Jg n ) * 9n) > -0(0-^ + °l) + ^n/16 , min Re MO- 

If p = or p = 1, then under the first condition (14. 6p the right-hand side here tends 
to +oo as n — > oo. If p > 2, it is bounded from below by 

a l -v(-c + a l J16 min Re MO) 

\(-r„\<a n 

where the expression in the brackets tends again to +oo . Therefore (6, {Jg n )*gn) > 
for sufficiently large n. Similarly choosing the sign " — " in (14.71) . we see that 
(6, (Jg n ) * g n ) < for sufficiently large n. □ 

Corollary 4.6. Instead of condition (14. 6p assume that 

lim Re&oo(£) = oo or lim Re&oo(0 = — oo. 

Then for both signs N±(b) > 1. 



24 



D. R. YAFAEV 



In contrast to Theorem 14.51 there are no restrictions in Corollary 14.61 on the pa- 
rameter p in the assumption b G C P (R)'. 

On the other hand, condition (14. 6p permits Re&(£) to tend to ±oo only on some 
system of intervals. Moreover, the lengths of these intervals may tend to zero. In 
this case, however, the growth of Re&(£) and the decay of these lengths should be 
correlated and there are restrictions on admissible values of the parameters p and I. 

Unlike Theorem 14.31 Theorem 14.51 does not guarantee that N = oo; see subs. 5.4, 
for a discussion of various possible cases. 

4.4. Theorem 14.21 can be combined with the Bochner-Schwartz theorem (see, 
e.g., Theorem 3 in §3 of Chapter II of the book \5\). It states that a distribution 
b G Co°(R)' satisfying the condition (b,Jg * g) > for all g G C£°(R) (such b are 
sometimes called distributions of positive type) is the Fourier transform 



for some x (that is, of at most polynomial growth at infinity). In particular, this 
ensures that b G S'. 

Theorem 14.21 implies that if (h, f * /) > for all / G P, then the corresponding 
distribution b is of positive type. It means that the sign-function s(x) of h(t) is 
determined by the measure dM(x): 



It is easy to see that condition ( 14 . 1 2 1) is equivalent to condition fll . 13j) on measure 
(I4.13p . In terms of distribution (II. lip , we have Xh\X)dX = dm(X). Therefore 
Theorem 12.91 leads to the following result. 

Theorem 4.7. Let h G Z' + and (h,f-kf) > for all f G V. Then h(t) admits the 
representation ( 11. 12ft with a positive measure dm(X), X G R+, satisfying for some k 
condition ( 11 . 13(1 . 

The representation ( 1 1 . 1 2 j) is of course a particular case of (ll.lOp . It is much more 
precise than ( 1 1 . 1 j) but requires the positivity of (h, f * f). Theorem 14. 71 shows that 
the positivity of (h, f * f) imposes very strong conditions on h{t). Actually, we have 




of a positive measure dM(x) such that 




(4.12) 




that is, s(x)dx = dM(x). Let us define the measure 

dm{X) = AdM(-lnA), AeK 



(4.13) 
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Corollary 4.8. Let h e Z' + and (h,f*f)>0 for all f G V. Then h G C°°(R+) 
and 

(-l) n h {n \t) > (4.14) 

for allt > and all n = 0, 1, 2, . . . (such functions are called completely monotonic) . 
The function h(t) admits an analytic continuation in the right-half plane Ret > 
and it is uniformly bounded in every strip Ret G (ti,t 2 ) where < t\ < t 2 < oo. 
Moreover, for some x G R and C > the estimate holds: 

h(t) < Cr 1 {l + |lnt|)* t>0. (4.15) 

All these assertions are direct consequences of the representation (I1.12p . In par- 
ticular, under condition (11.131) we have 

hit) < Cmax (e~' A A(l + I lnA|) x ) 

A>0 v ' 

which yields (I4.15p . 

Note that according to the Bernstein theorem (see, e.g., Theorems 5.5.1 and 5.5.2 
in [1]) condition (14. 14[) implies that the function hit) admits the representation 
(I1.12p with some measure dm(\). Of course, condition (14. 14j) does not impose any 
restrictions on the measure dm(X) (except that the integral (I1.12p is convergent for 
all t > 0). In contrast to the Bernstein theorem we deduce the representation (11.121) 
from the positivity of the Hankel form. In this context condition (11.131) is due to 
the assumption h G Z' + . 

We mention also a related result of H. Widom. He considered in [13] Hankel 
operators H with kernels h(t) admitting the representation (11.121) and showed that 
H is bounded if and only if m([0, A)) = 0(A) as A — > and as A — > oo. In this case 
hit) < Ct^ 1 for some C > 0. Thus Theorem 14.71 and Corollary 14.81 can be regarded 
as an extension of Widoms's results to unbounded operators. 

Under the positivity assumption the identity (12.211) takes a more precise form. 

Proposition 4.9. Let h G Z' + and (h, f * f) > for all f G V. Then there exists a 
positive measure dM(x) satisfying condition (14.121) for some x such that 



/oo 
u 1 (x)u 2 (x)dM(x) 
-oo 



for all fj G T>, j — 1,2, and Uj = G Z where the mapping H is defined by 

5. Applications and examples 

5.1. Consider first self-adjoint Hankel operators H of finite rank. Recall that 
integral kernels of Hankel operators of finite rank are given (this is the Kronecker 
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theorem - see, e.g., Sections 1.3 and 1.8 of the book [10]) by the formula 

M 

h(t) = Y,Pm(t)e- a ™ t (5.1) 

m=l 

where Rea m > and P m (t) are polynomials of degree K m . If H is self-adjoint, that 
is, h(t) = h(t), then the set {cti, . . . ,ajf} consists of points lying on the real axis 
and pairs of points symmetric with respect to it. Let Ima m — for m — 1, . . . , M 
and Im a m > 0, a Ml +m = a m for m = M + 1, . . . , M + M x . Thus M = M + 2M X ] 
of course the cases M = or M x = are not excluded. The condition h(t) = h{t) 
also requires that P m (t) = P m (t) for m = 1,...,M and PM^mif) = P m (t) for 
m = M + 1, . . . , M + Mi. As is well known and as we shall see below, 

M 

rank H = ^ K m + M =: r. 

m=l 

For m — 1, . . . , M , we denote by p m = p m the coefficient at t Km in the polynomial 

Pm(t). 

The following assertion yields an explicit formula for the numbers N±(H). Its 
proof will be given in [18] . 

Theorem 5.1. For m = 1, . . . , M , set 

ATj m) - 1 = A/l m) = K m /2 if K m is even and p m > > (5.2) 

A/"j m) = A^l m) - 1 = K m /2 if K m is even and p m < 0. J 

Then the total numbers N±(H) of (strictly) positive and negative eigenvalues of the 
operator H are given by the formula 

M M +Mi 

N ± (H) = Y,^± n) + E K m + Mi. (5.3) 

m=l m=Mo+l 

Formula (15.21) shows that every pair 

P m (t)e- amt + P m+Ml {ty~ am+Ml \ rn = M + 1, . . . , M + M u (5.4) 

of complex conjugate terms in (15. ip yields K m + 1 positive and K m + 1 negative 
eigenvalues. The contribution of every real term P m (t)e~ amt where m — 1, . . . , M 
also consists of the equal numbers [K m + l)/2 of positive and negative eigenvalues 
if the degree K m of the polynomial P m (t) is odd. If K m is even, then there is 
one more positive (negative) eigenvalue if p m > (p m < 0). In particular, in the 
question considered, there is no "interference" between different terms P m (t)e~ amt , 
m = 1, . . . , M , and pairs (15. 4p in representation (15. ip of the kernel h{t). 
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According to (15. 3p the operator H cannot be sign-definite if M\ > 0. Moreover, 
according to (15 .2j) . J\f±^ — for m — 1, . . . , M if and only if K m = and =Fp m > 0. 
Therefore we have the following result. 

Corollary 5.2. A Hankel operator H of finite rank is positive (negative) if and only 
if its kernel is given by the formula 

Olmt 



h(t) = ^ P™e 



m=l 



where a m > and p m > (p m < 0). 

Corollary 15.21 admits different proofs which avoid formula (I5.3p . For example, 
one can use that although the functions P m (t)e~ amt are analytic in the right-half 
plane Ret > 0, they are bounded for t = t + io~ &s a — > oo for a constant P m {t) 
only. Therefore according to Corollary 14.81 such Hankel operators cannot be posi- 
tive. Alternatively, using formula (15.161) below, one can deduce Corollary 15.21 from 
Theorem 14.51 

5.2. Consider now Hankel operators H with kernels (j!.14p . Since the case k = 
0, 1, . . . (finite rank Hankel operators) has been discussed in the previous subsection, 

here we suppose that k ^ 0, 1, If k > — 1, condition (II. 8p is satisfied for all k, 

and the operators H are compact (actually, they belong to much better classes of 
operators). If k = — 1, then condition (11.81) is satisfied for k > 1, and the operators 
H are bounded but not compact. 

Let us calculate the corresponding b- and s-functions. If k > —1, then function 
fl2TT()|) equals 

POD 

a(£) = (2tt)- 1/2 / t k e~ at r^dt = (27r)- 1/2 «- 1 - fc+i «r(l + k — z£), (5.5) 
Jo 

and hence function (12.181) equals 

»<« - ^' (T W^i- (5 - 6) 

If k — — 1, then in accordance with formulas (I5.5P and (15. 6p . we have 
a(n = (2 7 r)- 1 / 2 a i « lim r(e - if), 6(0 = (2n)- 1 a i k(^ + i0) _1 . 

It yields the expression 

s(x) = 0, x > /3, s(x) = 1, x < (3, where ft = — lna, (5.7) 



for the function s = y/2TT&*b. Formula (15. 7\i remains true for the Carleman operator 
C (the Hankel operator with kernel h(t) = t _1 ) when a = 0. Indeed, in this case 
according to ( I2.23P the sign-function s(x) = 1. 
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Next, we calculate the Fourier transform of function ( 15. 6ft . Assume first that 
k G (—1,0). Then (see, e.g., formula (1.5.12) in the book jl]) 



poo 

Jo 



r(i-<o 

Making here the change of variables t + 1 = a~ l e~ x , we find that 



Passing now to the inverse Fourier transform, we see that for k G (—1,0) the sign- 
function s(x) = Sf.{x) of kernel (11 . 14|) equals 

^^pAy^-^- 1 . (5.8) 

Let us verify that this formula remains true for all non-integer k. To that end, 
we assume that ( 15. 81) holds for some non- integer k > — 1 and check it for k\ — k + 1. 
Since 

r(i + fc 1 -zO = (^i-^)r(i + A;-zO, 

we have 

s fel (x) = a _1 (A;i - d)s k (x). 

Substituting here formula (15. 8p for Sk(x) and differentiating this expression, we 
obtain formula (15.81) for 3^(2). This concludes the proof of relation (I5.8P for all 
k > -1. 

Lemma 5.3. Let h(t) be given by formula (11.141) where k ^ Z + . Then the sign- 
function is determined by relation (I5.8p . 

Actually, relation (I5.8P remains true for k e Z + if one takes into account that the 
distribution [e~ x — a)+ fc_1 has poles at integer points. For example, for k = we 
have 

s( x ) = a- 1 5(x-(3). (5.9) 

Obviously, s(x) = for x > (3 = — Ina. If k = —1, then s(x) = 1 for x < (3. If 
k G (—1,0), then s(x) > and s G L X (]R). Therefore it follows from Theorem 14.31 
that H > 0. 

If k > 0, then distribution (15.81) does not have a definite sign. Therefore it can 
be deduced from Theorem 14.21 that the corresponding Hankel operator also is not 
sign-definite. 

Alternatively, for the proof of this result we can use Corollary 14.61 Formula (12.111) 
implies that function (I5.6P has the asymptotics 

6(0 = (2n)- 1 a- 1 - k ^(- l k (l + Ode!" 1 )), 1^1 -> 00. (5.10) 
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Making the dilation transformation in (II. 14ft . we can suppose that a = 1. Then we 
have 

Re6(0 = (27r)- 1 cos(7rA;/2)^ + O(^- 1 ), £ -> +oo, (5.11) 

Since cos(nk/2) ^ unless k is an integer odd number, this expression tends to ±oo 
if ±cos(7r/c/2) > 0. Thus Corollary 14.61 for the case bo = ensures that the Hankel 
operator H is not sign-definite. 

Let us summarize the results obtained. 

Proposition 5.4. The Hankel operator with kernel dl . 14[) is positive for k 6 [—1, 0], 
and it is not sign-definite for k > 0. 

Explicit formulas for the sign-functions can also be used to treat more complicated 
Hankel operators. For example, in view of (15. 7p the following assertion directly 
follows from Theorem 14.31 

Example 5.5. The Hankel operator with kernel 

h{t) =r\e- ait - ie ~ a2t ), 7 >0, 
is positive if and only if a 2 > oti > and 7 < 1. 

5.3. In this subsection, we consider the Hankel operator H with kernel (I1.15p . 
Condition (II. 8p is now fulfilled for all k, and the operator H belongs of course to 
the Hilbert-Schmidt class (actually, to much better classes). Observe that 

POO 

a(f) = (27T)- 1 / 2 / e~ tr r*dt = (2vr)- 1 / 2 r- 1 r((l - if)/r) 



and define, as usual, the function &(£) by formula (I2.18P so that 

m = w l IX Sr_% /r) - (5 ' 12) 

Consider first the case r > 1. It follows from the Stirling formula (12.111) that for 
all r > 1 the modulus of function (I5.12p exponentially grows and the periods of its 
oscillations tend to zero only logarithmically as |£| — > 00. Therefore Theorem 14.51 
implies that the Hankel operator with kernel (11.151) is not sign-definite. 

The Hankel operator H with kernel h(t) = e~ l can also be treated (see Appen- 
dix B) in a completely different way which is perhaps also of some interest. This 
method shows that both positive and negative spectra of the operator H are infinite. 

If r = 1, then h(t) = e~ l yields a positive Hankel operator of rank 1. 

Let us now consider the case r < 1. Then, again according to the Stirling formula 
(12. lip , function (15.121) belongs to L X (IR) so that its Fourier transform 

, r r((i-<e)/r)^ 



*(,-) = (27T/T' / f.(i-»o ^ =:/ - (x) (5 ' 13) 
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is a continuous function which tends to as \x\ — > oo. Therefore by Proposi- 
tion I4.4t he corresponding Hankel operator H > if and only if I r (x) > for all 

x et 

It turns out that I r {x) > 0. Surprisingly, we have not found a proof of this fact 
in the literature, but it follows from our results. Only for r = 1/2, integral (15.131) 
can be explicitly calculated. Indeed, according to formula (1.2.15) of [I] 

3fcM = 2 1 -^ 7 r-^r(3/2-iO. 

Therefore it follows from formula (I2.25f) that 

h/2 ( x ) = 2-\- l ' 2 e^ 2 e~ eX ' 4 (5.14) 

which is of course positive. 

For an arbitrary r G (0,1), one can proceed from the Bernstein theorem on 
completely monotonic functions (see subs. 4.4). Observe that if 

^(t) = r p e~ tr , p>0, (5.15) 

then 

if,'(t) = -pt- p - l e~ tr - rt- v + r - l e- tr . 

Further differentiations of ip(t) change the sign and yield sums of terms having the 
form (I5.15p . Thus the function h(t) = e~* satisfies for all n condition (I4.14p and 
hence admits the representation f l 1 . 1 2 j) with some positive measure dm(X). It follows 
from f H~T2|) that 



(Hf,f)= / |(L/)(A)| 2 dm(A)>0, V/eCj 



where L is the Laplace transform (12. 27|) . Since the operator H is bounded, this 
implies that H > 0. 

Thus we have obtained the following result. 

Proposition 5.6. The Hankel operator with kernel ( 11.151) is positive for r e (0, 1], 

and it is not sign-definite for r > 1. 

Putting together this result with Theorem 14. 3^ we see that integral ( I5.13P is pos- 
itive for all r e (0, 1). Our indirect proof of this fact looks curiously enough. 

5.4. Let us now discuss convolution operators with growing kernels &(£). We 
emphasize that condition (14.61) does not guarantee that the numbers N±(b) are 
infinite. Indeed, consider the kernel h(t) = t k e~ at where k is a positive integer. 
Formula (15. 6 P shows that for Ima = the corresponding function &(£) 

6(0 = (2vr)- 1 «- 1 -^(l -<£)•••(*- <0 (5.16) 

has a power asymptotics as |£| — > oo. According to Theorem 15.11 the positive and 
negative spectra of the Hankel operator H with the kernel h{t) are finite; for example, 
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H has exactly (k + l)/2 positive and negative eigenvalues if k is odd. Moreover, if 
Ima / 0, then in view of (15.161) the function exponentially grows as £ — > +oo 
or £ — > — oo. Nevertheless the Hankel operator H with kernel h(t) = t k {e~ at + e~ at ) 
has exactly k + 1 positive and negative eigenvalues. 

On the other hand, for kernel (I5.12p where r = 2 we have N±(b) = oo. This 
follows from Theorem 14.11 because, by Proposition IB.lt the Hankel operator with 
kernel h(t) = e _< has infinite number of positive and negative eigenvalues. 

A similar phenomenon occurs for Hankel operators with non-smooth kernels. This 
is discussed in the next section. 



6. Hankel operators with non-smooth kernels 

6.1. Let the symbol of the Hankel operator H be defined by the formula = 
e «*oM Evidently, u G R°° if to < 0, and hence H = in this case. If t > 0, then the 
integral kernel of H equals 5(t — 1 ) so that 

(Hf)(t) = f(t -t). 

Condition (13.1 2p is now satisfied for N = and k = 0. 

The operator H admits an explicit spectral analysis. Indeed, observe first that 
(Hf)(t) = for t > t and hence L 2 (t , oo) C KerH. Since H 2 f = f for / G 
L 2 (0, to), the restriction of H on its invariant subspace L 2 (0,to) m ay have only ±1 
as eigenvalues. Obviously, the eigenspace 1-L± of H corresponding to the eigenvalue 
±1 consists of all functions f(t) such that f(t) = ±/(t — t). Since 

n+®n-® L 2 (t , oo) = L 2 (R+), 

the spectrum of H consists of the eigenvalues 0, 1, —1 of infinite multiplicity each. 
In this example, the 6-function equals 

Note that all functions (15.121) where r > 1, (16. 1 p as well as ( I6.2p below exponentially 
grow and oscillate at infinity. In these cases the corresponding Hankel operators 
have infinite positive and negative spectra. 

6.2. It follows from Corollary 14.81 that a Hankel operator H can be sign-definite 
only for kernels h G C°°(R + ). Actually, if h{t) or one of its derivatives hS l \t) has a 
jump discontinuity, then H has infinite number of both positive A„ and negative 
— An~\ n = 1,2,..., eigenvalues, and we can even calculate their asymptotics as 
n — > oo. Positive (negative) eigenvalues are of course enumerated in decreasing 
(increasing) order with multiplicities taken into account. 

Let us start with an explicit 
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Example 6.1. Let h(t) = (t - *)' for some I = 0,1, ... ii t < to and hit) = if 
t > t . Then 

(Hf)(t)= [ (to-t-s) l f(s)ds, te(0,t o ), 
./o 

and (Hf)(t) = for t > to- For such /i(t), the symbol equals 

i 

/,•! 



fc=0 

and the 6-function equals 



6 «» = 2,r(/ + 2-ifl - (6 ' 2) 



Let us consider the spectral problem Hf = Xf, that is, 



f° (to-t-s) l f(s)ds = Xf(t), t 6(0, t ). (6.3) 
./o 

Differentiating this equation k times, we find that 

{-l) k l{l - 1) ■■■{I- k + 1) f '(to - t - s) l - k f(s)ds = A/( fe )(t) (6.4) 



for & = 1, . . . ,1. Differentiating the last equation where k = I once more, we see 
that 

l\f(t) = \(-l) l+1 f l+1 \t -t), t 6(0, t ). (6.5) 
Setting t = t in ( 16. 3 p and ( 16. 4p . we obtain the boundary conditions 

/(to) = f(to) = --- = / a) (t )=0. (6.6) 

Conversely, if a function f(t) satisfies equation ( 16. 5 p and boundary conditions ( 16. 6p . 
it satisfies also equation ( 16.31) . This leads to the following intermediary result. 

Lemma 6.2. Let the operator A be defined on the Sobolev class H /+1 (0,t ) by the 
equation 

(A/)(t) = (-l) m / (m) (to-t). (6.7) 
Considered with boundary conditions (16. 6p . it is self-adjoint in the space L 2 (0,t ), 
and its eigenvalues a n are linked to eigenvalues X n of the operator H by the equation 
a n = HX- 1 . 

6.3. Clearly, A 2 is a differential operator and the asymptotics of its eigenvalues 
is described by the Weyl formula. However, to find the asymptotics of eigenvalues 
of the operator A, we have to distinguish its positive and negative eigenvalues. For 
this reason, it is convenient to introduce an auxiliary operator A with symmetric 
(with respect to the point 0) spectrum having the same asymptotics of eigenvalues 
as A. 
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We define the operator A by the same formula (16. 7\\ as A but consider it on 
functions in H z+1 (0,to/2) © H l+1 (t /2, t ) satisfying the boundary conditions 

/«(0) = /M(t /2 - 0), /«(* /2 + 0) = /«(*„), (6.8) 

where k — 0, . . . , I for Z even and 

/W(0) = /«(t /2 - 0) = / (fc) (t /2 + 0) = /<*>(*„) = 0, (6.9) 

where fc = 0, ...,(/ — l)/2 for I odd. The operator A is self-adjoint in the space 

L 2 (0, t ) = L 2 (0, t /2) © L 2 (t /2, t ), 

and it is determined by the matrix 

X= {a 21 ^o 2 )' A ^ = A h> ( 6 - 10 ) 

where A 2 ,i ■ L 2 (0,t /2) -» L 2 (t /2,t ). The operator A 2; i is again given by relation 
(16 ,7p on functions in H i+1 (0,t /2) satisfying conditions (16. 8 p or (16. 9 p at the points 
and to/2 — 0. It follows from formula (I6.10p that the spectrum of the operator A 
is symmetric with respect to the point and consists of eigenvalues ±a n where a 2 
are eigenvalues of the operator A% ^2,1 =: A. 

An easy calculation shows that A is the differential operator A = (— \y+ 1 Q 2l + 2 
in the space L 2 (0,to/2) defined on functions in the class H 2 ' +2 (0, to/2) satisfying the 
boundary conditions f^ k \0) = f^ k \to/2) where k = 0, . . . , 21 + 1 for I even and the 
boundary conditions 

/(*>(0) = f^(t /2) = f 0+^(0) = j< l+1+k \t /2) = 0, 

where k = 0, . . . , (I — 1)/2 for I odd. The asymptotics of eigenvalues a 2 of A is given 
by the Weyl formula, that is, 

a n = (2TTt 1 n) l+1 (l + O(n~ 1 )). 

Let us now observe that the operators A and A are self-adjoint extensions of a 
symmetric operator Aq with finite deficiency indices. For example, Aq can be defined 
by formula (16.71) on C°° functions vanishing in some neighbourhoods of the points 0, 
to/2 and t . Therefore the operators A and A have the same asymptotics of spectra. 
Taking Lemma [6.21 into account, we obtain the following result. 

Lemma 6.3. Eigenvalues of the Hankel operator H defined in Example \Q.l\ have the 
asymptotics 

= l\(2n)- l -% +1 n- l -\l + 0{n- 1 )). 
Remark 6.4. In the case / = we have the explicit formulas 

AW = (27T)- 1 t (n-3/4)- 1 , Ai-) = (27T)- 1 t (n-l/4)- 1 , 71 = 1,2,.... 
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6.4. Now we are in a position to obtain the asymptotics of the spectrum of Hankel 
operators whose kernels (or their derivatives) have jumps of continuity. We combine 
Lemma [6731 with the result by V. V. Peller (see Theorem 7.4 in Chapter 6 of his book 
|10j ) which implies that singular numbers s n (V) of a Hankel operator V satisfy the 
bound 

s n (V) = oin- 1 - 1 ) 

if V has a symbol belonging to the Besov class Bi^L x(R). Applying the Weyl 
theorem on the preservation of the power asymptotics for the sum of operators, we 
can state the following result. 

Theorem 6.5. Let I G Z + , and let v(t) be the Fourier transform of a function in 
Bf+^OR). Set 

h(t) = h (t - t) 1 + v(t) 

for t < to and h(t) = v(t) for t > to- Then eigenvalues of the Hankel operator H 
have the asymptotics 

\^ = \h,\l\{2ix)- l -% +1 n- l -\l + o{l)) 

as n —7- oo. 

We emphasize that under the assumptions of this theorem the leading terms of 
the asymptotics of positive and negative eigenvalues are the same. Of course if h(t) 
becomes smoother (I increases), then eigenvalues of the Hankel operator H decrease 
faster as n — > oo. 

7. Perturbations of the Carleman operator 

In this section we consider operators H = Hq + V where Hq is the Carleman 
operator C (or a more general operator) and the perturbation V belongs to one of 
the classes introduced in Section 5. Different objects related to the operator H 
will be endowed with the index "0" , and objects related to the operator V will be 
endowed with the index ' V . 

7.1. For perturbations V of finite rank, we have the following result. 

Theorem 7.1. Let the sign-function Sq(x) of a Hankel operator H be bounded and 
positive. IfV is a Hankel operator of finite rank and H = H + V , then 

N-(H) = N_(V). 

In particular, H > if and only ifV>0. 

Applying Theorem 15.11 to the operator V, we get an explicit formula for the total 
number of negative eigenvalues of the operator H. 
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Corollary 7.2. Let the kernel v(t) ofV be given by the formula 

M 

v(t) = J2P m (t)e- amt 

m=l 

where P m (t) is a polynomial of degree K m . Define the numbers A^j; by formula 
(15. 2p . Then AL (H ) is given by formula (15.31) . 

Since for the Carleman operator C the sign-function equals 1, Theorem 17. II applies 
to H = C. 

The inequality N_(H) < N_(V) is of course obvious because H > 0. On the 
contrary, the opposite inequality AC (if) > N^(V) looks surprising because the 
operator Hq which may have the continuous spectrum is much "stronger" than the 
operator V of finite rank. At a heuristic level the equality AC (if) = N_(V) can 
be explained by the fact that the supports of the sign-functions s (x) and s v (x) are 
essentially disjoint. Very loosely speaking, it means that the operators H and V 
"live in orthogonal subspaces", and hence the positive operator if does not affect 
the negative spectrum of if. The detailed proof of Theorem 17.11 as well as that of 
Theorem 15.11 will be given in [18] . 

7.2. Let C be the Carleman operator, and let V be the Hankel operator with 
kernel 

v (t) = t k e~ a \ a > 0, k > — 1. (7.1) 

The operator V is compact, and hence the essential spectrum spec ess (if 7 ) of the 
operator 

H 1 = C - jV, 7 G M, (7.2) 

coincides with the interval [0, 7r]. Since the sign-function of the operator C equals 
1, the sign- function s 7 of the operator if 7 equals 

s 7 (x) = 1 — js v (x) 

where the function s v (x) is given by formula (15. 8p . 

Let first k G (—1,0). Observe that s v (x) is continuous for x < (3 = — In a and 
s v (x) — > +oo as x — » f3 — but s v G L X (R) Thus the function s 7 (x) —> — oo as 
x — > j3 — for all 7 > 0, and hence it follows from Theorem 14.31 that the operator 
if 7 has an infinite negative spectrum for all 7 > 0. 

In the case fc>0we use the formula 

b(o = m+uo (7.3) 

and apply Corollary 14.61 with 6 (£) = <K£) an d &oo(0 — ^(0- Since 60 G C(R)' and 
6oo has asymptotics (15.101) . the operator if 7 has a negative spectrum for all 7 7^ 0. 
Let us summarize the results obtained. 
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Proposition 7.3. Let iJ 7 = C — where V is the Hankel operator with kernel 
( HP) . Then: 

1° The operator H 1 has an infinite negative spectrum for all 7 > if ' k G (—1, 0). 
2° TTie operator has negative eigenvalues for all 7 7^ if k > 0. 

7.3. The result below directly follows from Theorem 14.31 

Proposition 7.4. Suppose that the sign-function s v (x) of a Hankel operator V is 
continuous and s v (x) — > as \x\ — > 00. Then the operator H 1 defined by formula 
(I7.2p is positive if and only if 

is v {x) < 1, VxG R. 

// t/iis condition is not satisfied, then H 1 has an infinite negative spectrum. 

We note that, by Proposition 13. 121 under the assumption of Proposition !7.4l on the 
sign-function s v the operator V is compact. Of course this assumption is satisfied if 
b v G L X (R). 

Example 7.5. Let v(t) = e~* r where r < 1. We have seen in subs. 5.3 that its 
sign-function s v (x) = I r (x) where I r (x) is integral (15.131) . Recall that I r (x) is a 
nonnegative continuous function of x G R and I r (x) — )■ as \x\ — )■ 00. Set 

i/ r = maxLfx). 

Then i/ 7 > if 7 < z/" 1 and the operator has infinite negative spectrum for all 
7 > v~ x . Using explicit formula (15.141) it is easy to calculate Vi/ 2 = 3^/Q/ire^ 3 ^ 2 . 

In the case r > 1 we use formula (17.31) . As shown in subs. 5.3, the modulus of the 
function exponentially grows and the periods of its oscillations tend to zero 

only logarithmically as |£| — > 00. Therefore Theorem 14.51 yields the following result. 

Proposition 7.6. Let v(t) = e~* where r > 1. Then the operator has at least 
one negative eigenvalue for all 7 7^ 0. 

Appendix A. Proof of Lemma [33] 

Set 

F^ n) = max ((lnt) K t n \F {n \t)\) 

where for shortness we use the notation (x) — (1 + |x|). 

Let us first consider ($*F)(A) for A G (-1, 1) =: /. We have 

2T($*F)(A)= / F(t)e lXt dt+ / F(t)e iXt dt, k = |A|~ 1/2 . 

JO Jk 
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The first integral in the right-hand side is bounded by Fq ' |A| _1 / 2 which belongs to 
In the second integral we integrate by parts 

POO poo 

/ F(t)e iXt dt = i\- l F(k)e iXk + I F\t)e iXt dt. (A.l) 

Jk Jk 

The first term here is bounded by C| A|~ 1 (ln X)~ k Fk°^ which belongs to L X {I) if k > 1. 
The second term is bounded by 



IAI 



POO 

1 / r^inty^dtF^ < C\X\- l (\n\)- K+l F^. 
Jk 



It belongs to L\I) if k > 2. 

Next, we consider ($*F)(A) for |A| > 1. Integrating by parts, we see that 



PK POO 

Vfr($*F)(\) = i\- 1 / F'^e^dt + iX- 1 F\t)e iXt dt. (A.2) 

JO Jk 

The first term here is bounded by 

IAI" 1 f t~ 1 {\nt)~ K dtFi K) < ClAr^lnA)^ 1 ^. 
Jo 

It belongs to \ /) if k > 2. In the second integral in ( 1A.2I) we once more 

integrate by parts, that is, we use formula (1A.1I) with F(t) replaced by F'(t). The 
function \- 2 F'(k) is bounded by A _2 A; _1 F (1) = |A|- 3 / 2 F (1) . For the second term, we 
use the estimate 

POO POO 

|A~ 2 / F"(t)e iXt dt\ < A- 2 / r 2 dtF {2) = A" 2 /^ 1 ^ = |A|- 3 / 2 F (2) . 

Jk Jk 

Therefore the second term in (IA.2I) also belongs to L X (]R \ /). 



Appendix B. The Gaussian kernel 

Here we return to the Hankel operator H with kernel h(t) = e~* 2 . Now we proceed 
from the identity 

(Hf,f) = (Qi>,4>), (B.l) 

where ip(t) = e - ' 2 f(t) and Q is the integral operator with real kernel e~ 2ts in the 
space L 2 (M + ). We shall use fIB.ip essentially in the same way as the main identity 
(11. 4p . Observe that operators Q with kernels q(ts) which depend only on the product 
of variables can be explicitly diagonalized by the Mellin transform (see [15]). Under 
fairly general assumptions on q the spectrum of Q consists of the interval [—7, 7] 
where 

7 = >/27rmax|(Mg)(0| 
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and M is the Mellin transform ( 12. 9ft . In particular, for q(t) = e 2t the spectrum of 
Q is absolutely continuous, simple and coincides with the interval [—^J-n/2, a/vt/2]. 
This allows us to check the following assertion. 

Proposition B.l. The Hankel operator H with kernel h(t) = e~* 2 has infinite 
number of positive and negative eigenvalues. 

Proof. Choose some /i G (0, yV/2). For an arbitrary N, let A^\ . . . , A^ C 

(fi, y/ir/2) and A\~\ . . . , A^ c (— a/tt/2, — fi) be closed mutually disjoint intervals. 

Choose functions ipf ] such that = E Q {Af ) )ipf ) and = 1, j = 1, . . . N. 

Let = XljLi Q; iV 9 j ± ' ) be a linear combination of the functions ip^ ,...,ip^ . 
Then 

N N 

±(g^),^)) = ±^| % | 2 (g^\^f ) )>^El«il 2 ll# ) ll 2 = * (±) ll 2 - (b.2) 

i=i i=i 

For an arbitrary e > 0, we can choose Vj ±} G C °°(M + ) such that H^-y^ll < e 

for all j = 1, . . . N. Since the functions tpj are orthogonal, the functions ipj are 
linearly independent if e is small enough. Moreover, it follows from (IB. 21) that 

±(g^ (±) ,^ (±) ) > 2-Vll^ (±) f 

if t/'W = X^Li c^'V^ an d £ is small. 

Set now f {±) {t) = e* 2 -0 (±) («- Th en G L 2 (R + ) and according to relation (|B~H 
we have the inequality ±(Hf±, f±) > on linear subspace of dimension N (except 
f± = 0). Hence the Hankel operator with kernel (I1.15P has at least N positive 
and N negative eigenvalues. Since N is arbitrary, this concludes the proof. □ 

We emphasize that the operator H is compact while the operator Q has the 
continuous spectrum. Nevertheless the multiplicities of their positive and negative 
spectra are the same (infinite). 
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